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This report is the year-end and final evaluation 

report for a federally funded bilingual education 

project, "Increasing the Quality of Basic Bilingual 

Instructional Services to K-3 Hispanic Children in 

West Texas," for the 1983-84 school year. The project 

was designed to organize and operate a bilingual 

education cooperative, monitored by the Education 

Service Center - Region 17 (Lubbock, Texas), so as 

to provide financial and technical assistance to 

three school districts in their efforts to provide 

higher quality bilingual instructional services to 

approximately 190 students. The three project schools 

were located in the public school districts of Plainview, 
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01 ton , and Petersburg, Texas. 

The overall goal of the project was to assist 
LEP (limited English proficiency) children to acquire 
and/or improve their English skills and using bilin- 
gual methodology and materials to facilitate and 
expedite writing, and content area concepts to the 
extent necessary to insure academic success in the 
regular school program. 

The external evaluation design for the 1983-84 
school year focuses on four general areas: the specific 
objectives of the project; a general question relat- 
ing to the overall impact of the project; the perceived 
effectiveness of the project personnel; and the overall 
goal of the project. 

With the exception of the objective pertaining to 
the effectiveness of the Parent Advisory Committee and 
degree of parental involvement, the objectives were 
satisfactorily achieved. The program impact statement 
was that the project did have a positive impact on the 
participating students. Specific program recommendations 
are included in the report. 
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FOREWORD 



The following evaluation report is for the bilinnual education project, 
"Increasing the Quality of Basic Bilingual Instructional Services to K-3 
Hispanic Children in West Texas," for the 1983-84 school year. This project 
was initially designed to operate during a three year period, and this 
evaluation report covers the third and final year of the project. 

Both internal and external evaluations have been major elements of this 
project. The internal evaluation was a continuing process conducted primarily 
by the. classroom teachers and the consultant from the Education Service 
Center, Region 17. The external evaluation function was conducted, • by B. E. 
Askins and Associates, Lubbock, Texas, which is an independent consultant and 
service organization with its direction primarily through various faculty members 
of the College of Education, Texas Tech University. 

This report, emanating from the external evaluation study, was attempted 
to be designed and written not only to present evaluation data for the 
respective decision-makers, but for any group of lay citizens who also may 
be interested in this project. 

The external evaluation recognizes and hereby expresses appreciation to 
administrators, parents, teachers, aides, and students for their excellent 
cooperation in this evaluation effort. 

The invaluable assistance of the various professionals and parapro.f essionals 
on the external evaluation team is also acknowledged. j 




B. E. Askins, Ed.D. 
External Evaluator 
B. E. Askins and Associates 



October, 1984 
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SECTION I 
INTRODUCTION 
Rationale of Bilingual Education 

The rationale of bilingual education programs as established under Title 
VII of the Elementary and Secondary Education Act of 1964 (Public ,Law 89-10) 
is that educational programs should be developed to meet the special educational 
needs of children who have limited English speaking ability, who come from 
environments where the dominant language is one other than English, and who 
come from low-income families. Some characteristics of bilingual education 
programs include: (1) English is recognized and taught as a second language 
to children whose dominant language is one other than English; (2) The children 
are taught one or more academic subjects in their dominant language, at least 
until they have mastered enough English to enable learning in English; 
(3) The children whose dominant language is English are taught the dominant 
language of the other children; (4) Provision is made for increasing the 
instructional use of both languages for both groups in the same classroom; 
and (5) The children are taught the history and cultural heritage which reflect 
the value systems of speakers of both languages. 

Using this rationale, the Education Service Center-Region 17 requested, 
and received, a grant from the U.S. Department of Education (ESEA, Title VII, 
Bilingual Education) to provide assistance to such students in West Texas 
beginning in the 1981-82 school year. A brief description of the Education 
Service Center is presented in the following, paragraphs . A description of 
the project as well as the evaluation data are presented in subsequent sections. 



Education Service Center - Region 17 



The Education Service Center - Region 17, located in Lubbock, Texas, is 
one of twenty service centers in Texss assisting local school districts in 
improving educational services to children. Assistance is provided through a 
variety of programs and services which include staff development, computerized 
data processing, a film library, instructional materials, a delivery service, 
and a variety of cooperative programs. Region 17 is located in the-South Plains 
of Texas. There are twenty (20) counties in the region with a total population 
of 412,000. There are sixty-four (64) school districts with a combined enrol- 
lment of 85,000 students of which 29,000 are enrolled in the region's only 
urban district. There are 31,000 students with Spanish surnames in the region. 

The geographical area served by the Education Service Center - Region 17 
is the leading Texas agribusiness area with farming and farm manufacturing 
employing the largest number of workers. The economic base includes cotton 
farming, oil production, ranching, manufacturing, and distribution. The 
majority of districts served are rural and relatively isolated. These factors 
are important in that they affect the range, types, and quality of educational 
services which are available to the students. The Education Service Center - 
Region 17 is the agency that serves in an intermediate role between the Texas 
Education Agency ;nd the sixty-four (64) districts. 

Organization of the Evaluetion Rep ort 

The organization of the remainder of this report includes the: 
Description of the Project; Evaluation and Design of the Project; Evaluation 
Data; and Summary, Conclusions, and Recommendations. 
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SECTION II 



DESCRIPTION OF PROJECT 

Overview of Project 

The federally funded project, "Increasing the Quality of Basic Bilingual 
Educational Education Instructional Services to K-3 Hispanic Children in 
West Texas," was initially started in the 1981-82 school year; thus, the 
1983-84 school year is the final year of this project. Basically, this project 
was designed to organize and operate a bilingual education cooperative, moni- 
tored by the Education Service Center - Region 17, so as to provide financial 
and technical assistance to three school districts in their efforts to provide 
higher quality bilingual instructional services to approximately 190 students. 

During the 1983-84 school year, this assistance included providing for: 
a project director (Bilingual Instructional Consultant); a project secretary; 
six (6) teacher aides; stipends (tuition, books, fees, and travel) for the" 
6 aides; instructional materials for the six (6) classrooms; resource materials 
for professional development/training for teachers, aides, administrators, 
and parents; consultants to assist in the training of the aides; and miscel- 
laneous supplies and services to support the project. 

Local of Project and Number of Students Served 

The school districts where the three elementary schools were located 
which participated in this bilingual education cooperative were: Plainview, 
Olton, and Petersburg. These school districts are located in the Llano 
Estacado region of West Texas whicl, has primarily an agricultural and industrial 
economic base. The three school districts are within a fifty mile radius of 



ll 



4 



Lubbock and are located in the northern section of the area assigned to the 
Education Service Center. The Mexican American student population within the 
three school districts range from approximately 45$ in Plainview to 60% in 
Petersburg and Olton. 

The number of students served by school and grade level during the 
1983-84 school year are as follows: 





K_ 


1_ 


2_ 


3_ 


Total 


P 1 a i n v i ew 


9 


11 


7 


6 


33 


Olton 


12 


22 


14 


14 


52 


Petersburg 


8 


8 


6 


5 


27 


Total 


29 


41 


27 


25 


122 



P arent Ad vis ory Comm ittee 

Tne primary purposes of the Parent Advisory Committee were to serve as: 
a group of lay advisors to the project, (2) the community support system for 
the various schools within the project; and (3) serve as an organization 
where training could be provided on the various aspects of bilingual education, 
objectives of the project, and on the role of the parent in the educational 
process of the student. 

O verall G oal of Project 

The overall goal of the project was to assist LEP (limited English 
proficiency) children to acquire and/or improve their English skills and 
using bilingual methodology and materials to facilitate and expedite 
writing, and content area concepts to the extent necessary to insure academic 
success in the regular school program. 
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Specific Objectives 

To achieve the primary or overall goal, as stated in the previous 
paragraph, four specific objectives were developed for the 1983-84 school 
year as related tc: student achievement, teachers and aides; the Parent 
Advisory Committee; and the development of a unique bilingual curriculum 
guide. The last objective was added during the 1983-84 year to facilitate 
curriculum adaption of existing materials in conjunction with the development 
of the bilingual education curriculum guides for each of the three project 
schools (Amendment to Project Application, dated 31 October 1983). These 
specific objectives are stated in the following paragraphs: 
Student Achievement 

1. At the conclusion of the 1983-84 project year, 70% of the LEP students 
participating will master at least 70% of the skills taught at their 
grade level . 

Teachers and Aides 

2. At the conclusion of the 1983-84 project year, 90% of the teachers and 
aides participating will increase their knowledge and competency in: 

A. Content area curriculum development 

B. OLD/ESl curriculum development 

C. Development of classroom materi al s for 
creative bilingual education 

D. Assessment 
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Parent Advisory Committee 

3. At the conclusion of the 1983-84 project year, 80% of parents in the 
Parent Advisory Committee will demonstrate an increased awareness, 
understanding and appreciation of the need for parental involvement in 
their children's learning process and the role of the Parent Advisory 
Committee in the implementation of the project. 

Bilingua l Curriculum Guide 

4. At the conclusion of the 1983-84 project year, a bilingual curriculum 
guide will have been developed for each of the three project school 

di stricts. 

Activities to Achieve t he Specific Objectives 
Various activities were identified to be performed in an effort to achieve 
objectives 1-4. A list of these activities (as included in the 1983-84 Con- 
tinuation Project Proposal .Applicati on) , by objective, are presented on the 
following pages (pp 7-10). 
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I. OBJECTIVE; At th<> conclus Ion of the 1 983-8^1 project year, 70% of the 
skills taught at their grade level. 



ACTIVITIES 



Tte teachers with the assistance of the 
aides will Instruct the LEP students In 
the six components of the Texss State 

1 . Basic concepts starting the student 

the school environment shall be taught 
In the student's primary language. 

2. Basic skills of comprehending, speak- 
ing, reading and writing shall be developed 
In the student's primary langutge. 

3. Basic skills ot comprehending, speak- 
ing, reading, and writing shall be devel- 
oped In the EngWsh language. 

4. Subjoct matter and concepts nhaH be 
taught In the study's primary language. 

5. Subject matter flrfd concepts shall be 
taught in the English language. 

G . Attention shall be given to Instilling In 
the student confidence, self-assurance • 4 
a positive Identify with his or her cultural 
heritage. 

The basic scop* will be In utilizing and 
Implementing a " Dlagnostlc/FYeccrlptlve 
approach to instruction. 



TO 
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TIME LINK 



M 



J 



LEP students participating will master at leant 70% o'f the 



PEBSONN EL J?ES FpNSI BLB 

Teachers with the assistance 
of aides. 



VALUATION 

All pre and posttests will be 
administered during ttie required 
time period In the spring. 

frogress/fVrforMsnce teste related 
to Hit* teaching units 

Project consultant plans for periodic, 
visitation to monitor s'jdetit progress 
ftntHfce implementation of teaching 
Strategies identified lil.the. objectives . 
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2. OBJECTIVE: At the conclusion of the 1983-84 project year, 90% of the teachers and aides participating will Increase their know ledye and 
competency In: 1) Content area curriculum development 
2) OI.D/ESL curriculum development 

2) Development of classroom materials for creative Bilingual Education; and 
4) Assessment 



ACTIVITIES 



Workshops 

1. Content area Instruction 
In Spenish and English 

2. Oral language development/ 
ESL trtethodologkes 

3. Ualng and making materials 
for the Bilingual classroom 

Conferences 

16 semeifcr hours college course 
work 

Design/conduct sharing and 
planing sessk>nt Witt) teachers, 
aides and administrators on effective 
ways to: 

1 . D*o«jnose special student 
needs 

2. Prescribe Activities to crw»<?t 
such needs 

3 . Intergrate culture 

OS 



TIME LINE 
M 



x x 



PERSONNEL RESfo WS[6LE, 



Project Consultant 

Outside consultants specialfeln? 
in Bilingual Education methodblogl** 



I. U.K. 

Project Consultant 



Rout tests 



Reports 



Transcripts 

Activity Imports 
Records of weekly 
visits and methods 
of coordination 
tfecord of meeting* 
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3. 



ACT]VITi£9. 



Quarterly P.A.C. meetings: 

1 . Parent on-site vtelt to 
project schools 

2 , Nutrition awareness for 
pi rents 

3.. "Basic Math Skills" 
assist parents in devel- 
oping activities In basic 
math aKIIIfl 'or their 
children 

4, The parent's nole In the 
classroom 



to 

m 
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TIME 1 
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NU 



M 



Project Consultant 
Coordinator of Migrant/ 
Bilingual 



EVALUATION 



Minutes of meetlngu 
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4. At the conclusion of the 1983-84 project year, a bilingual 

three project school districts. 



ACTIVITIES 



Curriculum Adaptation 

-Adaption existing materials 
on the basis of the 
linguistic and cultural 
characteristics of the 
Child 

-Adapting materials to meet 
the LEP child's educatlona 
needs 
-Plan approach to 
development of curriculum 
guides . (3 days) 

Developing Materials: 

-Interdisciplinary materials 
CO -Developing evaluative 
criteria to determine 
appropriateness of 
materials 
-Evaluating materials, 
commercial and 
?s»» teacher-made 

-Preparation of first draft of 
curriculum guides . (3 days) 
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Review and edit drafts. (2 days) 



Final Review & edit 



Publish approved Bilingual 



^pj^loului^ guides 



TIME LINE 



D 



J 



M 



M 



1 1 



guide will have been developed for each of the 



PERSONNEL 
RESPONSIBLE 



Project Consultant, Outside 
Consultant and Project 
Personnel 



EVALUATION 



Project Consultant, Outside 
Consultant and Project 
Personnel 



Project Consultant, Outside' 
Consultant and Project 
Personnel 

Project Consultant, Outside 
Consultant 



Project Consultant 
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SECTION III 

EVALUATION AND OESIGN 
Purpose of External Evaluation 
The purpose of the external evaluation function was to establish and main- 
tain a procedure of collecting and providing information for decision-making 

t 

relative to student and program progress. The external evaluation function 
was conducted by B. E. Askins and Associates, Lubbock, Texas. This is an 
independent -consultant and service organization with its direction primarily 
through "various faculty and staff members of the College of Education, Texas 
Tech University. 

Evaluation Design 

The external evaluation design of this project was developed and stated 
in the original project proposal; however, due to some operational changes in 
the project and that the present external evaluator did not get involved with 
the project until the latter part of the 1983-84 school year, some modifications 
to the evaluation design were necessary. In the opinion of the evaluator, the 
evaluation design as described in the following paragraphs will better answer 
the necessary evaluation questions as related to the project. 

The external evaluation design for the 1983-84 school year focused on 
four (4) general areas: the stated specific objectives of the project; the 
overall goal of the project; a general question relating to the overall impact 
of the project; and the perceived effectiveness of the project director. 
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Specific Objectives 

Four (4) specific objectives were stated in an effort to achieve the 
overall goal of the project, and the complete statement of these objectives 
are presented in Section II. The ways in which these objectives were evaluated 
are presented in the following paragraphs: 

Objective 1 - Student Achievement . This objective was evaluated by: 
reviewing and analyzing the students' assigned year-end grades (report cards) 
for each grade level at the three project elementary schools; reviewing and ana 
lyzing scores from the achievement tests; reviewing the informal records and 
work of the students; interviews with the teachers (and aides) at the project 
schools and informal classroom visits. 

Objective 2 - Teachers and Aides . This objective was evaluated from data 
obtained from: a review of records indicating number of college/university 
semester hours completed. 

O bjective 3 - Parent Advisory Committee . This objective was evaluated by: 
a review of the PAC yearly' report and informal interviews with the project 
director and administrators and the three building principals. 

Objective 4 - B i lingual Curriculum Guide. This objective was evaluated by 
reviewing the completed curriculum guide; obtaining informal reactions on the 
guide from the teachers and administrators; and reviewing th.e report submitted 
by the project director. 

""X 

Program Impact 

The question designed to attempt to assess program impact was: Are the 
outcomes of program participants (students) different from what would have 
occurred in the absence of the program? 
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An answer to this question was sought by two avenues: a review ana 
analysis of the student profile data, and review and analysis of test scores 
from standardized achievement tests in the areas of English and Spanish 
language development. 

Student Prof i le Data . Each project teacher maintained a profile on each 
LEP student in her class. Information on placement and test scores was enter- 
ed on the form throughout the year. At the end of the year, each teacher 
rated each student in six areas: 

1. Attitude toward his/her native language 

2. Attitude towards the bilingual program 

3. Attitude toward school 

4. School attendance record 

5. Library use 

6. Teacher observation of classroom behavior 

Each topic was rated on a Likert scale form (1-poor to 5-excel lent) . The 
results were calculated and reported in percentages form by school and grade. 

Achievement Test Scores . Two standardized achievement tests were 
administered in grades 1-3. The Metropolian Test was used to measure English 
achievement (reading and language arts) and the Comprehensive Test of Basic 
Skills - Espanol was used to measure Spanish achievement (reading and language 
arts). These tests were administered in each of the three project schools on 
a spring to spring basis. Also, these same tests were administered in grades 
1-3 in three other schools which served as a control group for comparison 
purposes in this assessment. Means and significance of the scores were reported 
by school, grade, and content area. 
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P roject Personnel 

The effectiveness of the various project persons was evaluated in a variety 
of ways. The nine (9) teachers were evaluated by the project director using a 
Likert scale form; the six (6) teachers aides were evaluated by their respective 
teacher in narrative form; and the project director was evaluated by each of the 
nine teachers using a Likert scale form. 
Over all Goal 

Based upon the general findings concerning the specific objectives, program 
impact, and effectiveness of the project personnel, a conclusion was drawn con- 
cerning attainment of the overall goal of the project. 
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SECTION IV 
EVALUATION DATA 

The external evaluation design for the 1983-84 school year focused on 
four (4) general areas: the stated specific objectives of the project; teach- 
ers and aides; a general question relating to the overall impact of the 
project; and the perceived effectiveness of the project personnel. 

Specific Objectives 

The four specific objectives which pertained to: (1) Student Achievement; 
(2) Teachers and Aides; (3) Parent Advisory Committee; and (4) development of a 
Bilingual Curriculum Guide. A statement of these objectives and evaluation 
data are presented in the following paragraphs. 
Student Achievement 

1. At the conclusion of the 1983-84 project year, 70^ of the LEP students 
participating will master at least 70% of the skills taught at their 
grade level . • -- 

This objective was evaluated by reviewing and analyzing the students' 
assigned year-end grades by content area, for each gra^e" level at the three 
project schools; reviewing and analyzing scores from achievement tests; review- 
ing the informal records and work of the students; and interviews with the 
teachers at the project schools and informal classroom visits. 

The best indicator for this assessment was the review and analysis of 
the assigned grades at the three project schools. 

Table 1 presents summary data indicating the percentage of students at 
the three project schools receiving 70% or more in the content areas of: 
reading, spelling, language arts-English, math, social studies, and science. 
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Table 1 

Percentage of Students at the Three Project Schools Receiving 
Satisfactory Grades in the Six Content Areas 



Reading Spelling Lang-Eng Math Social Studies Science 



School 


N 


% 


% 


% 


% 


% 


% 


Olton 


65 


95 


98 


91 


89 


92 


94 


PI ai nvi ew 


33 


100 


84 


93 


88 


100 


84 


Petersburg 


23 


87 


88 


71 


82 


94 


76 


Total 


121 
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Although the percentages were generally high for each grade at the three 
schools, these were certain areas where the 70% factor was not achieved. 
These were at Plainview in 63 in language arts-English (67%) and science (33%) 
and Petersburg in G2 IN SPELLING (60%) and G3 in language arts-English (20%), 
math (40%) , and science (20%) . 

Concl usi on . Based upon these data and the informal visits to the class- 
rooms, there is ample evidence to conclude that objective #1 was satisfactorily 
achieved. 

Teacher s and Aid es 

2. At the conclusion of the 1983-84 project year, 90% of the teachers and 
aides participating will increase their knowledge and competency in: 
Content area curriculum development; OLD/ESL curriculum development; 
development of classroom materials for creative bilingual education; 
and assessment. 

This objective was evaluated from data obtained from: a review of 
records indicating the number of college/university semester hours completed 
by the aides; in-service program reports; and a review of the agenda of the 

in-service sessions. 

28 
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During the 1983-84 project year, the six teacher aides completed a total 
of 81 college/university semester hours (152 semester hours completed a total 
3-year project). Also, four of the six aides completed at least 18 semester 
hours during the 1983-84 year. 

During the 1983-84 project year, eighteen in-service training sessions 
were conducted for the teachers and aides by the Education Service Center. 
The topics for these sessions included: Bilingual Education/ESL Guidelines; 
Texas Administrative Code-Chapter 75 and Bilingual Education; Instruction of 
Essential Elements in the Native Language; Techniques for Teaching Transition/ 
ESL; Bilingual Special Education; Language Arts in Spanish; Correlation of TAC 
Chapter 75, Essential Elements and Adopted Textbooks; and Transfer of Skills. 

In addition, the teachers and aides received training in language arts, 
methodology, and demonstrated mastery of the skills present in the area by 
developing a curriculum guide for grades K-3. Other training in the areas of 
culture, English as a Second Language (ESL) and content area instruction was 
offered to the teachers and aides. Implementation of these skills in the 
classroom provided support to the instructional component of the project. On- 
site training was also conducted by the project director (who served as a 
bilingual instructional consultant to the project). The teachers demonstrated 
mastery of bilingual instructional methodology by following up on demonstra- 
tion classes and continued use of materials and strategies. Documentation of 
these activities can be found in the activity reports and school service 
reports. 

Conclusion. Based upon these data, it was concluded that objective #2 
was satisfactorily achieved. 
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Parent Advisory Committee 

3. At the conclusion of the 1983-84 project year, 80% of the parents in 
the Parent Advisory Committee will demonstrate an increased awareness, 
understanding and appreciation of the need for parental involvement in 
their children's learning process, and the role of the Parent Advisory 
Committee in the implementation of the project. 
This objective was evaluated by a review of the PAC yearly report and by 
conducting informal interviews with the project director, teachers, and the 
building principals. 

Records indicated that the Parent Advisory Committee (PAC) consisted of 
eight members and formally met three times during the 1983-84 school year. 
The PAC participated in the project as advisors. The members were kept 
informed of the activities of the project and were often solicited for com- 
ments and recommendations relative to the bilingual education project. 
Training was provided to the PAC during the formal meetings on the various 
aspects of bilingual education and che role of the parent in the educational 
process' of the student. According to the year-end PAC report, "this group 
acted as the community support system for the school." 

Concl usi on . Evaluating this objective proved to be very difficult be- 
cause of the many subjective and judgemental elements. The PAC report was 
positive about the accomplishment of this group for 1983-84 and all project 
personnel, including the building principals, fully supported the concept and 
rationale for the PAC. However, during the visits to the project schools, the 
evaluator sensed a deep concern of the teachers about the lack of parental 
support or involvement of the parents with the educational process of their 
children. Whether the PAC could have worked more on this problem is unknown, 
but there was a definite concern by the teachers. Although the operation of the 

30 



V 



19 



PAC undoubetdly assisted the eight committee members, the effort d i d * not go far 
enough and it must be concluded that this objective was not achieved.- 
Bilingual Curriculum Guide 

4. At the conclusion of the 1983-84 project year, a biYvrtgual curriculum 
guide will have been developed for each of the project school districts. 

This objective was evaluated by: reviewing the completed curriculum guide; 
obtaining informal reactions concerning the guide from the teachers and admini- 
strators; and reviewing the report submitted by the project director. 

A work task group consisting of seven project teahcers, the project di- 
rector, and a consultant (Dr. Maria Rivas) from Texas Tech University was 
formed as part of the Curriculum Development Project. The tasks of this group 
included : 

1. Translate into Spanish the language arts essential elements (as pre- 
scribed in the recently mandaded Texas Administrative Code - Chapter 75 
for grades K-3. 

2. Develop objective's in Spanish for each of the descriptors/sub-descrip- 
tors of the esential elements. 

3. Develop a correlation between the objectives from the essential 
elements and the Economy Spanish Reading Series . 

4. Develop an all-inclusive scope and sequence incorporating the essential 
elements (TAC-Ch75) and the reading series used by the school districts. 

5. Identify activities which can be used to teach the objectives developed. 

6. Implement the scope and sequence developed and use the first. year 
(1984-85) to refine, edit and adapt to fit the needs of the bilingual 
students in the three school districts. 
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The unique bilingual education curriculum guide has been completed and is 
now in the' process of being refined, edited, and adapted (Task #6). The, work 
group should be complimented for an outstanding job because such a document is 

greatly needed by all school districts. This unique curriculum guide will be an 

i 

asset to the instructional project in these schools for many;jyears. 

Concl usi on . Based on the above informantion, there is ample evidence to 
conclude that objective #4 was satisfactorily completed. 

Program Impact 

The question designed to attempt to assess program impact was: "Are the 
outcomes of program participants (students) different from what would have 
occured in the absence of the program?" 

An answer to this question was sought by two measures: a review and analy- 
sis of the student profile data and review and analysis of test scores from 
standardized achievement tests in the areas of English and Spanish language 
development. -'" 

Student Profile Data 

Each project teacher maintained a profile form on each LEP student in her 
class. At the end of the year, each teacher rated each student in six areas 
which were as follows: 

1. Attitude toward his/her native language 

2. Altitude toward bilingual program 

3. Attitude toward school 

4. School attendance record 

5. Library use 

6. Teacher observation of classroom behavior 
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The ratings were done on a five-point Likert style scale. A rating of five 
(5) indicated that the student's attitude was excellent" in a particular area. 
A rating of one (1) indicated a poor attitude. The results were tabulated for 
each project school by grade and are presented on Tables 2 through 4. The 
figures are percentages for each grade in the three project schools - Plainview, 
Petersburg, and Olton. Tables 2 through 4 are presented on the following pages. 
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Table 2 

Student Profile Data 
* Plainview 

Excellent 



Attitude toward native language: 

K 
1 
2 
3 

Attitude toward bilingual program: 

K 
1 
2 
3 



Attitude toward school 



School Attendance 



K 
1 
2 
3 



K 
1 

2 
■3- 



K 
1 
2 
3 



Teachers observation of classroom 
behavior : 

K 
1 

2 
3 



Library Use: 



64 
14 



48 
29 



36 
71 



27 
14 
33 



36 
14 



36 
57 



4 3 
(Percentages) 



27 
57 



36 
71 



36 
14 



73 
57 
33 



18 
43 



54 

28 
33 



9 

28 
100 



9 

100 



27 
14 
16 



28 
33 



45 
43 



9 

14 
33 



Poor 



67 



50 



16 



50 



33 
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Table 3 



Student Profile Data 
Petersburg 



23 



Excellent 



Poor 



Attitude toward native language: 

K 
1 
2 
3 



Attitude toward bilingual program: 

K 
1 
c 
3 



Attitude toward school 



K 
1 
2 
3 



29 
38 
20 



14 
37 



29 
50 

40 



4 3 
(Percentages) 



29 
13 
40 
80 



57 
12 
40 
60 



29 
37 
40 
40 



14 
38 
20 
20 



50 
40 
40 



14 

40 
20 



29 
13 
20 



14 
20 



12 
20 



14 



29 



School Attendance: 



K 
1 
2 

r.3 



14 

50 
20 
60 



25 
20 
40 



57 
12 
40 



14 

12 
20 



14 



Library Use: 



K 
1 
2 
3 



25 
40 



14 

37 
40 
60 



57 
37 
40 



14 
20 



14 



Teachers observation of classroom 
behavior: 

K 
1 

2 
3 



14 " 
50 

40 



4? 
12 
40 
40 



14 
37 
40 
20 



14 

20 



14 



c 
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Table 4 
Student Profile Data 
01 ton 
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Attitude toward native language: 

K 
1 
2 
3 



Attitude toward bilingual program: 

K 
1 
2 
3 



Attitude toward school: 



School Attendance: 



K 
1 
2 
3 



K 

,1. 
•2 
3 



Library Use: 



K 
1 
2 
3 



Teacher observation of 
classroom behavior: 



K 
1 

2 
3 



Excellent 



Poor 



4 3 
(Percentages ) 



58 
50 



15 

22 



19 
28 



8 



50 
55 



31 
16 



19 

28 



58 
55 



19 

28 



15 
17 



8 



51 
78 



23 
16 



11 
6 



81 
61 



15 
28 



54 
50 



31 
33 



15 
17 
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The data reported on the Student Profiles generally indicate that students 
were rated as having very positive attitudes in most areas. This positive trend 
seems to hold true for all schools and at most grade levels. The exceptions 
are not especially noticable. There were nu consistent poor ratings in any of 
the schools, and improvement was noted over last years' ratings reports in the ■ 
1982-83 evaluation report. 

Achievement Test Scores , : 

Two standardized achievment tests were administered in grades 1-3. The 
Metropolitan Achievement Test was used to measure achievement in English (read- 
ing and language arts) and the Comprehensive Test of Basic Skills - Espanol was 
used to measure achievement in Spanish (reading and language arts). These tests 
were administered in each of the three project schools (Plainview, Olton and 
Petersburg) on a spring to spring basis. Also, these same tests were admini- 
stered in grades 1-3 in three other schools, which served as a control group 
for comparison purposes in the assessment. Schools in the control group were 
Lockney, Muleshoe, and Idalou. 

Evaluation of the test scores included a comparison of the project schools 
with the control group schools by grade and content area. This typeof 
comparison should allow a determination to be made about program impact. If 
the project schools have significantly higher scores than the control schools, 
it is reasonable to cofK^Uide that the program is probably having the desired 
effect. 

As a point of reference, mean scores are reported for each sub-test for 
each grade along with a probability level (p). The probability level indicates 
the degree to which the differences between scores may be due to program impact 
or some other factor. If a p value of .05 or smaller is reported, it is 
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probably reasonabjft^tp assume that the program is having the desired effect. 
The Mann-Whi tne^p^eft^a cpmmonly used statistical procedure with small 
groups such as those reported here, was utilized to make the comparisons 
between schools. 

Names of the project schools as compared with control schools are: 
Plainview-Lockney; OUon-Muleshoe; and Petersburg- Idalou. The comparative data 
are presented in the following paragraphs and in Tables 5, 6 and 7, which are 
presented on the following pages. 
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Table 5 

Comparison of Achievement Test Scores 
for Plainview and Lockney 

i 1 



Grade 


N 


Plainview 


Lockney 


N 


p* 






Total Reading-English 




- 


K 


0 


No Cases 


No Cases 


0 


0 


i 


22 


38.45 


42.37 


16 


.32 


2 


14 


29.50 


28.00 


18 


.84 


3 


14 


36.50 


39.87 


16 


.32 






Total Language-English 






K 


0 


No Cases 


No Cases 


0 


0 


1 


20 


29.80 


34.29 


17 


.43 


2 


14 


28.35 


31.66 


18 


.60 


3 


14 


38.92 


35.37 


16 


.39 






Total Reading - 


Spanish 






K 


0 


No Cases 


No Cases 


0 


0 


1 


20 


46.40 


No Cases 


0 


0 


2 


14 


40.71 


No Cases 


0 


0 


3 


14 


34.14 


No Cases 


0 


0 






Total Language - 


Spani sh 






K 


0 


No Cases 


No Cases 


0 


0 


1 


0 


No Cases 


No Cases 


0 


0 


2 


0 


No Cases 


No Cases 


0 


0 


3 


0 


No Cases 


No Cases 


0 


0 



*2 tail probability 
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Table 6 



Comparison of Achievement Test Scores 
for Olton and Muleshoe 



Grade 



K 
1 
2 
3 



0 

22 
14 
14 



01 tor, 



Muleshoe 



Total Reading - Engl ish 



No Cases 
38.45 
29.50 
36.50 



No Cases 
42.37 
28.00 
39.87 



Total Language - English 



0 
16 
18 
16 



K 


0 


No Cases 


No Cases 


0 


0 


1 


20 


29.80 


34.29 


17 


.43 


2 


14 


28.35 


31.66 


18 


.60 


3 


14 


38.92 


35.37 


10 


.39 






■ - Total Reading - 


Spani sh 






K 




No Cases 


No Cases 


0 


0 


1 


20 


46,40 


No Cases 


0 


0 


2 


14 


40.71 


No Cases 


0 


0 


3 


14 


•34.14 


No Cases 


0 


0 






Total Language - 


Spani sh 






K 


0 


No Cases 


No Cases 


0 


0 


1 


0 


No Cases 


No Cases 


0 


0 


2 


0 . 


No- Cases \ 


No Cases 


0 


0 


3 


0 


No Cases 


No Cases 


0 


0 



*2 tail probability 
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Table 7 

Comparison of Achievement Test Scores 
for Petersburg and Idalou 



Grade 


N 


Petersburg 


Idalou 


N 


p* 






Total Reading 


- English 






K 


5 


46.80 


No Cases 


0 


0 


1 


5 


65.80 


54.33 


3 


.036 , 


2 


4 


32. 25^ 


51.28 


7 


.042 


3 


5 


31 .40/ 


40.00 


4 


.19 






l ota l] Language 


- tng i i s n 






K 


5 


49.4CK 


No Cases 


0 


0 


1 


5 


74. ,00 


38.33 


3 


.05 


2 


4 


22.00 


51.28 


7 


.05 


3 


5 


40.40 ' 


46.00 


4 


.06 






■ -'' Total Reading - 


Spani sh 






K 


0 


No Cases 


No Cases 


0 


0 


1 


5 


51.60 


No Cases 


0 


0 


2 


3 


8.0 


No Cases 


0 


0 


3 


3 


9.0 


No Cases 


0 


0 






Total Language - 


Spani sh 






K 


0 


No Cases 


No Cases 


0 


0 


1 


0 


No Cases 


No Cases 


0 


0 


2 


0 


No Cases 


No Cases 


0 


0 


3 


0 


No Cases 


No Cases 


0 


0 



*2 tail probability. 
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PI ainvi ew -Lockney. Table 5 lists mean scores on the Metropolitan Achieve- 
ment test (MAT) and the Comprehensive Test of Basic Ski 1 Is- Espanol (CTBS) for 
Plainview, which is the project school, and Lockney, which is the control school. 
Scores reported for Plainview are generally higher than those reported for 
Lockney. Differences in total reading scores for the second grades, and total 
language scores for the two kindergartens are highly significant. This suggests 
that in these two grades, so far as the test scores are an indicator, the 
project is having a positive effect. Very few scores were reported on the CTBS 
for either school. This of course makes it impossible to draw any conclusions 
about program impact in these two areas. 

Olton-Muleshoe . Scores reported in Table 6 reflect performance in Olton, 
the project school, and Muleshoe, the control school. There does not appear to 
be any statistically significant difference between the scores in the two 
schools and few scores were reported for Spanish, it would be difficult to 
draw any definitive conclusions from these data. The scores from the schools 
are substantially similar-.-' Any variations are probably due to chance. 

Peters burg-Idalou. Table 7 compares the Petersburg project school with 
Idalou which served on the control school. There is a statistically significant 
difference between scores in grades one or two in total reading and total 
language. In both cases, the first grade at Petersburg has a higher mean score 
than Idalou. The situation is reversed for scores in the second grade. Again, 
this would suggest a positive program impact in the first grade at Petersburg. 
It is difficult to interpret the significantly higher scores for second grades 
at Idalou, It is possible that this difference reflects discrepant abilities 
in the two groups prior to testing. \ 

Concl usi on . Determine an answer to the question, "Are the outcomes of the 
program participants (students) different from what would have occured in the 
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absence of the program," also proved to be very difficult because of incomplete 
test data and many judgemental factors. Also, using the three school as a 
control group as a basis to compare the effectiveness of the project schools 
was probably not an appropriate evaluation design strategy for this project. 

In addition, determining program effectiveness based upon test scores from 
standardized tests acquired from a spring to spring test cycle is a questionable 
evaluation practice in this type of instructional program. However, based upon 
the data obtained concerning the student profiles and the achievement test 
scores, it was concluded that the outcomes of the students participating in the 
program would, in fact/ have been different than if they had not participated 
in such a program. Th$t"1s, the program did have a general positive impact on 
the participating students. 

Perceived Effectiveness of Project Personnel 

As part of the external evaluation, an effort was made to determine the 
perceived effectiveness of' the various personnel of the project. That is, how 
did the teachers perceive the effectiveness of their aides, how did the teachers 
perceive the effectiveness of the project director, and how did the project 
director perceive the effectiveness of the teachers. 

The perceived effectiveness of the various personnel was assessed in a 
variety of ways. The teachers evaluated their aides using narrative statements; 
the teachers evaluated the project director using a 6-item Likert scale form 
(5-1); and the project director evaluated the teachers using a 10-item Likert 
scale form (10-1) . 

As concerns the presentation of personnel evaluation data, it was decided 
that it was unnecessary to neither use names nor describe the items on the 
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evaluations instruments, but rather to present summary findings of the three 
groups of personnel . 
Teacher Aides 

Overall, the teachers perceived their aides as being very effective in the 
classroom. Some strengths reported included: works well with the LEP students; 
ESL instruction; served as good model for language acquisition; provided > 
excellent reinforcement in the Spanish language while working in content areas. 
Some perceived weaknesses included: weak in written language, both English and 
Spanish; weak in English Grammar, both oral and written; and weak in some areas 
of math. 
Teachers 

Overall, the teachers were perceived as being extremely effective. On the 
10-item scale, four received 10's, four received 9's, one 8, and one received a 
9 rating. In short, the project director appeared to be very satisfied with the 
effectiveness of the project teachers. 
Project Director ■ 

Overall, the teachers perceived the project director as being extremely 
effective as indicated from the high ratings (all 5's and 4's) on the evaluation 
form. Also, many narrative comments were noted on the form, such as: — always 
eager to give assistance, and share her knowledge with us; — was always willing 
to bring needed materials or provided me with information when I needed it; — 
always willing to demonstrate teaching skills in my class. In short, the 
teacners had much respect for the project director and considered her to be 
very effective in her position. 

Overall Goal 

The overall goal of the project was to assist LEP (limited English profi- 
ciency) children to acquire and/or improve their English skills and using 
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bilingual methodology and materials to facilitate and. expedite writing, and 
content area concepts to the extent necessary to insure academic success i v 
the regular school program. 

Based upon the general findings concerning the four specific objective, 
program impact, and the effectiveness of the project personnel, there appeared 
to be ample evidence to conclude that the overall goal of the project was 
achieved during the 1983-84 school year. 
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■ SECTION V 

" SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 

Summary 

This is an evaluation report for a federally funded bilingual 
education project, "Increasing the Quality of Basic Bilingual Instruc- 
tional Services to K-3 Hispanic Children in West Texas, "for the 1983-84 
school year. 

This project was designed to organize and operate a bilingual 
education cooperative, monitored by the Education Service Center-Region 
17, so as to provide financial and technical assistance to three school 
districts in their efforts to provide higher quality bilingual instruc- 
tional services to approximately 190 students. This assistance included 
providing for: a project director; a project secretary; six teacher 
aides; stipends for the aides; instructional materials for the six 
classrooms; resource materials for professional development/training for 
teachers, aides, administrators, and parents; consultants to assist in 
the training of the aides; and miscel laneous supplies and services to 
support the project. The three elementary schools which were considered 
as the project schools were located in the public schools districts of 
Plainview, Olton, and Petersburg, Texas. 

The overall goal of the project was to assist LEP (limited English 
proficiency) children to acquire and/or improve their English skills and 
using bilingual methodology and materials to facilitate and expedite 
writing, and content area concepts to the extent necessary to insure 
academic success in the regular school program. 
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To achieve the overall goal, four specific objectives were developed 
for the 1983-84 school year which pertained to: (1) student achievement; 
(2) teachers and aides; (3) the Parent Advisory.. Committee; and (4) develop- 
ment of a unique bilingual education curriculum guide. 

The external evaluation design for the 1983-84 school year focused 
on four general areas: the specific objectives of the project; a general 
question relating to the overall impact of the project; the perceived 
effectiveness of the project personnel; and the overall goal of the project. 
A summary of the conclusions based upon the evaluation findings in these four 
areas are presented in the following paragraphs. 

Conclusions 

Based upon the findings as presented in detail in Section IV, the 
following conclusions were developed. 
Specific Objectives 

Objective 1 . At the.concl usion of the 1983-84 project year, 70% of 
the LEP students participating will master at least 70% of the skills 
taught at their grade level. 

Based upon the findings, there was ample evidence to conclude that 
objective #1 was satisfactorily achieved. 

O bjective 2 . At the conclusion of the 1983-84 project year, 90'X of 
the teachers and aides participating will increase their knowledge and 
competency in: content area curriculum development; OLD/ESL curriculum 
development; development of classroom materials for creative bilingual 



47 



education; and assessment. 

Based upon the findings, there was ample evidence to conclude that 
objective #2 was satisfactorily achieved. 

Objective 3 . At the conclusion of the 1983-84 project year, 80% of 
the parents in the Parent Advisory Committee will demonstrate awareness, 
understanding and appreciation of the need for parental involvement in 
their children's learning process, and the role of the Parent Advisory 
Committee in the implementation of the project. 

Based upon the findings, it was concluded that objective #3 was not 
satisfactorily achieved. 

Objective 4 . At the conclusion of the 1983-84 project year, a 
'bilingual curriculum guide will have been developed for each of the three 
project school districts. 

Based upon the findings, there is ample evidence to conclude that 
objective #4 was satisfactorily achieved. 
Program Impact • 

The question to determine program impact was: "Are the outcomes of 
the program participants (students) different from what would have occur- 
red in the absence of the program?" 

Based upon the findings, it was concluded that the outcome of the 
students participating in the program would have been different if they 
had not participated in such a program. That is, the program did have a 
general positive impact on the participating students. 
Perceived Effectiveness of Project Personnel 

Based upon the findings, all personnel in the project (teachers, 
aides, and the project director) were perceived as being effective in their 
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respective roles. 
Overall Goal 

Based upon the general findings concerning the specific objectives, 
program impact, and effectiveness of the project personnel, it was con- 
cluded that the overall goal of -the project was achieved during the 
1983-84 project year. 
Additional Observations 

Although not based on specif ic measurable outcomes, the following 
observations are submitted as part of the external evaluation process: 

1. Instruction was provided in the child's native language by 
bilingual teachers and teacher aides at each of the three 
project schools. 

2. The project teachers were provided instructional assistance 
by a teacher aide at the three project schools. 

3. Instructional bilingual materials were provided to the 
project schools" by the Education Service Center. 

4. Minority role models were provided for the students at 
the three project schools. 

5. The effort being made by the teacher aides in obtaining 
academic degrees and teaching certificates will ultimately 
have a positive impact on the school and community at each 
of the project sites. 

6. The project director served as an excellent support system 
for the teachers and aides and served as a communication 
vehicle to the administration at the three project schools. 
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Recommendati ons 

Based upon the findings and conclusions of this evaluation study, 
the following recommendations are submitted: 

1. That efforts be made by the Education Service Center and the 
cooperating school districts to continue to provide some type of assis- 
tance to this group of students and teachers even though federal assis- 
tance is no longer available. 

« 

2. That a Parent Advisory Committee at each project school be 
organized and operated. 

3. That the bilingual curriculum guide as initially developed as 
part of this project be continually refined and made available to all 
schools serviced by the Region 17 Education Service Center. 

4. That, if control groups are necessary in future evaluation 
designs of bilingual education programs, the control group be identified 
within the same school of the project being compared or evaluated. 

5. That, when possible, standardized testing to measure student 
achievement be conducted on a fall-spring basis rather than on a spring- 
spring basis. This would undoubtedly yield a fairer measure of the 
effectiveness of the instructional program. 
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